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ABS'^RACT 

The issue raised by this paper is whether the site 
adfliinistratox needs to have formal evaluation skills. These skills 
are not necessary when there is a competent administrator and a 
competent evaluator who uses his or her skills within the context of 
a harmonious relationship with the school staff* However, the rarity 
of such a relationship, canpled with growing state demands for 
evaluation of categorically funded programs, indicates that today's 
principal needs to have some knowledge of the field of evaluation. 
There are a minimum of ten basic areas that the administrator should 
have a det-ailed knowledge of: evaluation design, needs assessment, 
aoal settina, obiectives, sampling techniques, philosophy of 
evaluation, student progress, testing^ computer printouts, and record 
keeping* In addition to the desirability of a harmonious working 
relationship between evaluator and administrator, public schools 
would profit from those in decision-^makincf positions having seme 
certification that wo'^.ld document their ability to relate effectively 
on evaluation issues* (Author/JK) 
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It has often been observed by program evaluators that a large number 
of the operating staffs of schools (e.g., those principals and other ad- 
ministrators who have been in the system for a large nimfiber of years) are 
without the requisite skills to understand and develop educational program 
evaluation. In earlier years, there may have been no necessity for those 
individuals to understand evaluation or even be cognizant of a few basic 
rudiments of the art, or as some evaluators hope, will soon be recognized 
as a profession. 

In recent years, a majority of large and small public school systems 
in the United States have been a party to or have been recipients of federal 
or state funding for special programs in the schools. This automatically 
dictates a mandate for evaluation. Evaluation in itself should be a vital 
part of the instructional process, which of course in the final analysis 
is dedicated to the enhancement of the educational well being of the chil- 
dren entrusted to the system. Program directors and/or principals probably 
are not sufficiently aware of what evaluation is. As a matter of fact, 
neither teachers nor principals, historically, have been previously employed 
in evaluation or are skilled at it, and perhaps we might ask why should they 
be? 

"Many teachers and administrators obviously are not managerial in 
their orientation and are afraid to engage in what they funda- 
mentally believe education to be. Otherwise they would not spend 
as much time as thej do turning out what can only be considered 
to be nuts and bolts produced according to technological specifi- 
cations. 

Principals and/or project directors usually have not been trained in 
evaluation; however, it is reasonable to assume that it would be desirable 
that those persons occupying leadership positions in public education have 
at least minimal amount of this training in their background, whether it 
be in teacher education, graduate work, or in-service training, to at least 
acquaint them with the basics in three or four key areas of evaluation. Too 
often, the school principal delegates evaluation responsibilities to another 
person at his site. Too often, also, the. district evaluator is content to 
have minimal contact with the project director-principal. 

"I need to point only to the use of paraprof essionals tn large 
urban systems to indicate what I mean. The term suggests that 
the system itself is being cautiously extended to include a few 
people without the traditional certification. My observations 
have taught me that the ones they have brought into the system 
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have had the juice of life squeezed froio them. Instead of being 
able to develop new educational approaches on their own or to 
make use of their own experiences in setting up creative learn- 
ing centers they are cut off from their peers and made to serve 
as agents of the experts in charge. Inevitably, they take on the 
coloration of professionalism and become more and more cautious 
the longer they work with those they think of as superiors, they 
begin to resemble some of the union leaders in that their loyalty 
is offered mainly to the structure of the system: they cease to 
be accountable to either the children or the community, 

The observation of those in the field is often that precisely the above 
is true. The administrators are quite often cut off fxom the mainstream of 
what it is they are all about; namely, the education, the quality education^ 
of children. The advancement of their own careers is a very human trait, but 
it does often lead them to divest their loyalties to the children, the school, 
and the community, and direct this loyalty to the structure of the system ard 
their own possible advancement within it. The problem then is, it is per- 
ceived in the field that there are in all probability a good number, perhaps 
even a majority, of site administrators or project directors who are (1) not 
abreast of even the basic fundamentals of evaluation; (2) perhaps may not be 
Interested In these fundamentals, but may be more interested In the advance- 
ment of their own careers. Both of the above are conditions that are to be 
expected in the contemporary scene. However, it is postulated that the public 
educational system would be well enhanced with those in decision-making positions 
having as a very minimal experience one or two courses in their basic training 
related to evaluation; better yet, to have some sort of certification at a 
point in their professional careers which would document the necessary expertise, 
or at least limited expertise, for them to relate effectively on evaluation 
issues to the program evaluator, to the staff of their schools, to the parents 
and the general public to which all of us in public education are ultimately 
responsible. 



Program Evaluations Skills Required 
for Educational Leadership 

In the first case we describe a particular situation in which the principal 
of a school played a key role in the evaluation/decision-making process over 
a period of three years. In this instance, a school district in California during 
the early 19708 created what was then a new and innovative idea, a career edu- 
cation high school. This school was known as the Harry S. Truman Career Educa- 
tion aigh School. Like other schools in the Hillside Unified School District, 
Truman High School fell under the supervision of the secondary schools division, 
but as elsewhere in the district, the responsibilities for general operations 
at the site level were delegated to the site principal. There were no special 
district regulations pertaining to Truman's career program despite its origin 
In the district offices. The then principal of the school said "the school 
district relationship is one of agreeable remoteness." Therefore, taking this 
statement into account, it was inc\jmbent upon the principal to run her school 
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in a- partial vacuum (this was a rare instance where a woman was assigned 
as principal of a predominately male-oriented activity). The principal 
was in charge of the entire staff and all program activity. She, in turn, 
was responsible to the secondary school division and periphery to the 
career education unit. 



Because the career program, which was called "outreach," took in the 
whole school, the principal was also entitled Title IV-C Project Director. 
At the time of this particular case study. Title IV-C was one of the federal 
government s main sources of funding for special projects in education. As 
such, a requirement for evaluation was a vital part of acceptance of these 
federal funds. The role played by Mrs. Johnson, the principal of the school, 
was no more than a reflection of her institutional position. She was solely 
responsible for the course of' study and the staff at Truman. In addition, 
she was the point of contact between the school and the district, the com- 
munity, state, and federal funding services. Mrs. Johnson had spent most 
of her educational career in Hillside Unified School District at the secon- 
dary level as a teacher and as an administrator of both junior high and 
high schools. She immediately created the impression of a well organized, 
confident, and thoroughly professional "take charge" kind of person. For 
Johnson, personal involvement seemed to be the core of her definition of 
the administrator's role in the school. Rejecting the idea of the^rincipal , 
the kind of faceless neutral decision maker, receiving information and hand- 
ing down orders, Helen Johnson insisted on participating in the educational 
process which took place around her. She left no doubt that hers was the 
ultimate responsibility when the decision had to be made, but she made it 
equally clear that such decisions would be based on her numerous personal 
observations and conversations with the staff and evaluator and a considera- 
tion of site personnel, as well as other factors. 

Mrs. Johnson's attitude was no more evident than in her work with 
Dr. Harry Emerson, who was the district evaluator assigned to the school, 
in the yearly evaluations of Truman High. Mrs. Johnson involved herself 
directly with the planning and implementation of evaluation procedures, and 
she served as the chief conduit of evaluative information between Dr. Emerson 
and the Truman stiff. An experienced administrator and thns no stranger to 
the increasing role of evaluations in public schools, Johnson maintained a 
balanced view of their real value. She conceded that the results of tne 
norm-referenced state achievement test could be used to improve some aspects 
of the school program; at the same time, she questioned the emphasis put on 
those tests, especially considering the amount of time they required. She 
also saw a lack of test wiseness among students of the lower socio-economic 
backgrounds common at this particular school, thus perhaps leading to 
invalid test scores. On the whole, Johnson dealt with program evaluation 
from a neutral c:autious position: "All of the test instruments are as good 
as the program and the evaluator." 

Mrs. Johnson thought that Dr. Emerson and his evaluation of "outreach" 
were very good indeed. She was particularly pleased with the attention 
Emerson gave to Truman High School, as exemplified by the number of site 
visits and his attendance at in-service workshops. She felt that Emerson 
included her in the important aspects of evaluation and because of this 
relationship, the evaluation of "outreach" was an asset rather than some- 
thing about which to be apprehensive. In fact, principal Johnson stated 
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that as far as she concerned, there was no way the evaluation could have 
been any better, "Dr. Emerson presented a very fair and very favorable 
picture of this project and outreach got more than its money's worth from 
his work." Interestingly enough, though, Johnson also noted that, for her 
purposes > the "gut level feeling" she got from being at Truman everyday 
and talking to people was as useful an evaluation as the formal written 
testing Icinds of assessments. 

This, then, raises an interesting point. Does the site admlpistrator 
in fact need to have detailed or even superficial knowledge of evaluation? 
In the case described, the evaluator himself was an expert in the field of 
career education and had a reasonable amount of formal evaluation skills. 
He also had a personality which blended well with site administrators and 
teachers and largely through this he was able to create an effective process 
evaluation throughout the three years of his association with the school. 
Therefore, it is possible in this particular case that evaluation skill on 
the part of the site administrator was not the important factor. The fact 
that she was an outstanding administrator within the school system was the 
Important consideration. The fact, also, that the relationship between her 
and the evaluator was almost symbiotic left little doubt that the evaluation, 
both process and summative, would be a success. The principal was able to 
entrust the day-to-day evaluation operation to Dr. Emerson although she 
required or requested frequent feedback on his findings. However, since, in 
her eyes, he was perceived as being competent at his craft and she was an 
extremely able admir.istrator , the two could work together each within an 
area of expertise, knowing full well that the other person would execute his 
or her end of the relationship in a professional manner. 

In this case, where there was a competent administrator and a competent 
evaluator trained in a specialty for which the school was designed, there 
was really no need for the administrator to have specific evaluation skills 
as long as the evaluator assigned to the school was himself competent in 
these skills. At the same time, if he had a personality that would lend 
itself to the utilization of these skills within the context of a harmonious 
relationship with the school staff, the evaluation would help the program 
succeed. This is not to say that ^he principal would not be well advised to 
have some basic evaluation skills at her grasp^ because within the bureau- 
cratlc system that employs her^ transfers occur almost yearly or within 
several years to other schools and other situations. The principal In this 
instance^ while she found herself within the hands of a capable evaluator, 
might find herselt two years in the future as a principal of a different 
school with a different program with a different evaluator and in a situa- 
tion in which she must have some basic knowledge of what an evaluation is, 
what statistics are-, what test scores are, and certain other tools which are 
described in the next case.^ 

The second evaluation we describe I's of an entirely different project; 
a different type of project in tJhich evaluation skills on the part of the 
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site administrator are considered essential. Here, also, the evaluator 
himself is in a position to influence the decision maker even though the 
administrator should have some knowledge of evaluation techniques. If 
evaluators can influence even a few educational decisions so that they turn 
out well, then this is surely a significant contribution. The prospective 
evaluators who harbor an image of themselves riding in like a white knight 
to an evaluation situation where the docile decision maker x^ill meekly 
follow the evaluator 's admonitions need to recognize that is often a fairy- 
tale view of the world. ^ 

Ihree critical decision points exist in the majority of educational 
evaluations, since most evaluations are performed for categorically funded 
programs such as Title I, the federal program to raise the basic skill level 
of low-achieving youth, or the School Improvement Program, which is a 
California state-funded activity primarily at the elementary level and is 
designed to raise the performance of all children in the school. The first 
decision-making point then is with the federal or state agency which dis- 
tributed the funds. Here, the evaluation may be divided between formative 
and summative issues. The state has certain reports it requires and it 
also often conducts on-site visitations during financial support of the 
proj ect. 

The next level for political decision making is within the district 
structure itself. If the district employs a competent and experienced edu- 
cational evaluator, his expertise can virtually be anticipated to lead the 
design and implementation of the evaluation with full cooperation of district 
officials. The competent evaluator should be given full authority and 
responsibility for program recommendations.' The wise evaluator, however, 
will always realize his position as just that — one of an evaluator. While 
he may have full authority for making recommendations, it is ultimately the 
project director who must make the final decision and implementation. The 
third decision-making point, after tie state and district-level staff, is 
the school principal. For internal purposes, this is the crucial one for 
process decision making. The personal characteristics of both the principal 
and the evaluator come into play heavily at this point. If the evaluator 
has demonstrated competence and at the same time is able to project a low- 
key but cooperative profile, he can probably emerge as a key person in pro- 
gram operation and changes in emphasis. In the hierarchy of politics of 
evaluation, the program evaluator's influence can probably, be the most 
deeply felt in his day-to-day relationship with the project operating staff. 
The evaluator would be well advised to content himself with successes at 
this end.-* 

In a secondary school district or a unified school district Including 
both elementary and secondary schools, the principal is given considerable 
autonomy in the operation of the programs within his or her school. In 
this type of evaluation which deals with Title I or School Improvement 
programs, the principal of the school as the primary site administrator in 



^Popham, W. J., Education Evaluation , Englewood Cliffs, NJ: Prentice Hall, 
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fact Is responsible for everything that goes on within the confines of his 
domain. If he then finds himself as the leader of a group of people involved 
in a categorically funded program, it obviously behooves the competent and 
conscientious administrator to have some knowledge of evaluation although, 
as previously mentioned, he would hopefully have a good evaluator who would 
ultimately do the actual collection of data. Nevertheless, the principal 
in today's school system definitely needs some knowledge of what is going 
on in the field of evaluation since it is almost assured that he or she will 
eventually become the head of a school involved in a categorically funded 
program. 

This relates to the nature of large city school districts where principals 
are rotated periodically to give them knowledge of the entire city and the 
community which they serve and to broaden their perspectives. Therefore, 
given the fact that an aspiring administrator or presently situated adminis- 
trator will, in fact, become involved in a state or federal program during 
his or her career, there are a minimum of ten basic areas in which this 
person should have, if mit expertise, at least detailed knowledge of what 
is being discussed and a basic concept of how to approach the area; to wit: 

!• Evaluation Desi? ,n; The design of the evaluation in many federal/ 
state programs is very veil circumscribed by government regulations. How- 
ever, within the broad regulations there is considerable latitude as to 
how to establish the objectives of the program, what levels to set the 
objectives, how to measure the objectives, and things of that nature. 
Therefore, the basic concept of what an evaluation design entails should be 
in the tool kit of school principals. Also, it might be noted that some 
school district have developed well designed "inhouse" evaluation reports 
that go far beyond the basic requirements. 

2- Needs Assessment : In order to establish the goals or the objectives 
of the school. It is first necessary to establish what the needs are. There- 
fore, a principal needs to understand the methods of collecting data to 
establish these needs. This might be through questionnaires, informal inter- 
views with teachers and parents in any number of methods. This is basically 
a site responsibility and should be conducted by the site administrator or 
by the site administrator's staff under his direction. 

3. Goal Setting ; Closely related to the needs assessment is the 
establishment of the level of performance to be expected from students in 
the school. Different schools, being in diverse parts of a large city, 
clearly draw from different populations. The expectancy in one part of the 
city due to socio-economic factors and other considerations might be cou- . 
si'ierably different from another part of the city. The principal, therefore, 
must be attuned to the situation in which he finds himself so that he may 
know how to set realistic goals for his school, ultimately to be evaluated, 
but he must also be cognizant that he should not be content with the exist- 
ing situation, but should continually strive to upgrade the performance of 
his students. He therefore must know how to establish a realistic level of 
expectancy. 

4. Objectives ; Objectives are one of the basic keys to any evalua- 
tion. They provide the direction to which the program is going. For example, 
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the objective reading might be tliat ''sixth-grade youngsters on the aver- 
age will achirive ten months' growth for ten months' instruction as measured 
by a norm-referenced test." The site administrator must know how to write a 
proper objective which includes all of the elements of the criterion level, 
who is to ob':ain it, to what degree, and under what conditions. 

^- Sampling Techniques: The administrator should have a smattering 
of knowledge as to how and why sampling is done. For example, in any large 
school having perhaps 500 Title I target students, it is not reasonable to 
examine the records of each and every subject. Therefore, it is incumbent 
to establi!;h a valid method of sampling through various class levels, various 
techniques of random selection, snd so on. Site administrator should defi- 
nitely understand how and why the evaluator samples students or classrooms 
as he doej . 

^- Philosop hy of Evaluation : The administrator should understand why 
projects are being evaluated. They, of course, are being evaluated because 
of a manc.ate from the funding source; however, there is much more to it 
than that:. Hopefully, the competent evaluator employed by a good school 
district will try to enhance the program within the schools to which he is 
assigned, and his evaluation itself will become part and parcel of the 
total irstructional process. The administrator needs to understand the 
basic philosophy that evaluation consists of more than just completing a fev 
state forms at the end of the year and saying that is that. It is, in fact, 
supposed to provide day-to-day or weekly or monthly feedback to the school 
on the progress of the educational program in relation to students. 

7. Student Progress : Analogous to this, how does evaluation relate 
to stucient progress? Do principals understand that it enhances student 
progre.'is? How can it be utilized in this direction? How can the evaluator 
assist the school, assist the teachers, and assist the administrators who 
ultimately will assist the students? 

8. Testing: Testing is a vital part of any program receiving outside 
funding. It is a part of our society, we might say, where we measure one 
persoi against another or we establish standards to which people should 
aspira. For example, all students in the United States might be tested on 
a given reading test and we might determine that the mean score is 30, 
Therefore, if there was a valid sample of all students, we could reasonably 
assuDie that 30 was what would be expected of an "average" student and this 
is wliat is called a norm^ref erenced or standardized test. These are 
required by all funding agencies, and in addition to that., most school dis-- 
tricrs have testing programs of their own, aside from categorical funding. 
Therefore, it is quite necessary that the site administrator understand 
what testing is about, how to interpret the results, and how to read the 
Computer printouts depicting those results. 

9- Computer Printouts : They come in various forms, often difficult 
for the lay person to understand, and in this sense we are calling the site 
administrator a lay person because, after all, he is not normally a pro- 
fessional evaluator. These printouts have the student's name, student's 
test: score, and oftentimes the same test score compared with how the student 
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did a year previously so the growth or the nongrowth may be determined 
ijDQSaed lately. The site administrator muse understand these pieces of student 
data that are given to him in order to interpret them for planning, for the 
faculty, or for the parents. 

RecQgu Keepings Record keeping is essential to evaluation. The 
teacher should not be burdened with excessive record keeping; however, it 
Is clearly impossible to measure student progress without record keeping; 
It is also ijnpossibie £01 the evaluator to determine the attainment of 
objectives without some records to examine. Therefore, the principal must 
establish within his or her school a system of record keeping that is simple 
yet adeqfjate to perfora the task, not only assisting with evaluation, but 
even more irapcrtantly , assisting the teacher in her instruction of the 
children and having an adequate record of each child's progress. 



Summary 



We have attempted to show the need for some knowledge on the part of 
school administrators relative to evaluation techniques. We have also tried 
to emphasize the desirability of a close and harmonious working relationship 
between evaluator and administrator. Apropos of the former point ie a quote 
from a recent work in the field of career education: 

"As a consequence of this growing emphasis on accountability and 
career education, career education project directors and practi- 
tioners need to become familiar with the evaluation measures and 
materials that are most appropriate to their particular 
circumstances.'*^ 

This view is expressed by a national figure in the field of career edu- 
cation. Its inherent meaning is valid for all fields of publicly supported 
education. In this arena, the term "accountability" should mean "a wllling- 
uesa to evaluate educational programs objectiveTy and express the results 
to (the public with candor." Towards this endj the Nevaluationwise adminis- 
trator can be of immeasurable help. 



^cCaslin, N. L. et al.. Career Education Measures; A Compendium of 
Svalu^tlon Instruments, Columbus, Ohio: The National Center for Research 
ixk Vocational Education, 1979. 
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